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Field of the Invention 

The present invention relates to the field of interactive television, and, more specifically, 
to synchronizing and customizing interactive content. 

Background of the Invention 
10 Delivery of interactive content with broadcast programming presents several special 

problems. First, for the interactive content to be effective, it must be synchronized with the 
broadcast. This entails delivering an interactive complement to an advertisement at the same 
time that the broadcast advertisement airs, or coordinating interactivity with events in the ad. 

Second, the delivery of interactive content must be reliable. Since the interactive content 
15 is broadcast in a data stream parallel to the video stream, there is the possibihty that the 
interactive content might be stripped out or corrupted at the broadcast site, or at the local 
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subsystem, such as a local network affiliate or a cable headend. Finally, it is often desirable to 
target the interactive content to the specific viewer or delivery system. Digital cable systems, for 
example, have much higher bandwidth for delivering interactive content than the limited 
bandwidth provided by the vertical blanking interval (VBI) data transmission methods available 
5 to analog cable systems. It is advantageous to build a system that selectively utilizes this higher 
bandwidth when present in the broadcast environment. Additionally, demographic factors might 
motivate targeting different interactive content to different delivery systems. 

A first order approach, as described in co-pending appUcation 09/333,724, entitled 
"Automatic Control of Broadcast and Execution of Interactive AppHcations to Maintain 

10 Synchronous Operation With Broadcast," filed June 15, 1999, assigned to the assignee of the 
present application and which is hereby incorporated by reference, involves the placement of an 
interactive broadcast server (IBS) at a broadcast facihty to insert interactive content into a video 
stream. This server utiKzes the network playlist system or other program identifying techniques 
in order to insert the appropriate interactive content for the current broadcast program or 

1 5 segment. A broadcast facihty using this technology is illustrated in Figure 1 . 

The broadcast facility uses various equipment along a broadcast facility transmission path 
to generate, enhance, modify, and transmit a video stream. In one embodiment, a video tape 
player 146 generates a video stream 145 firom video sources 1 12, and places the stream 145 on 
the transmission path. A server module (software application) running on the IBS (interactive 

20 broadcast server) 104 receives real-time information on the current video program being 
broadcast from a playlist control system 108. The playlist control system 108 controls the 
sequencing of the video sources 1 12, which are typically pre-recorded video content (such as 
programs and advertisements). The IBS 104 uses the playlist information to retrieve an 
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interactive content and uses a data insertion unit 1 16 to insert a retrieved interactive content or 
data into the video stream 148. In some systems, the sequencing of pre-recorded video content is 
performed manually or program information is pre-recorded on the video and detected by the 
IBS 104 (for example, through VBI encoding of an interactive content identifier). 
5 In an alternate embodiment of this interactive content delivery system, tapes are pre- 

prepared with interactive content and placed in the video source. A video tape player 146 plays 
the prepared tapes to generate a video stream with interactive content, and the interactive content 
is then delivered to the local subsystem 124. In this embodiment, the interactive content is pre- 
stored in a vertical blanking interval, for example, in VBI line 15, and an IBS 104 may not be 

10 used. In either embodiment, the combined video and interactive content stream 148 is then 

delivered to a communications system 120 (typically via satellite), to a local subsystem (such as 
a local network affiliate) 124 where the interactive content is then either delivered over the air 
through an antenna 149, to an interactive customer premise equipment (CPE) 128, i.e. a set top 
box or similar device, or over a cable system 149 that re-retransmits the local signal to a CPE 

15 128. 

While the above system can provide both synchronization and data reliability, some 
problems remain in ensuring that interactive content reaches its final destination due to broadcast 
facility or local subsystem equipment used along the transmission path. For example, many 
networks have multiple feeds to provide time shifted delivery of content for appropriate time 
20 zones. The equipment used by the broadcast facility to record and replay video content for 

different time zones may inadvertently delete, corrupt, or otherwise effectively modify (corrupt) 
the inserted interactive content. Also, broadcast facilities may have the ability to visually shrink 
a broadcast signal to allow the broadcaster to insert their own video content, e.g., a stock ticker, 
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into the broadcast signal This insertion process may destroy the interactive content that is 
already encoded in the broadcast signal. Further, a broadcast facility may convert an analog 
broadcast signal to provide digital signals to subscribers. However, as the analog-to-digital 
conversion equipment is typically located downstream of the point where interactive content is 
added to the video stream, the conversion equipment may corrupt the interactive content encoded 
in the signal. In general, any device that modifies the video stream may corrupt or destroy 
interactive content. 

Additionally, local subsystems 124 may also inadvertently corrupt inserted interactive 
content along a local subsystem transmission path, similar to the broadcast facility transmission 
path. Video manipulation in both paths can result in the loss of interactive content. For 
example, a local subsystem may choose to time shift a video stream by a half-hour to smt its own 
programming choices. In this situation, the equipment used to tape and delay the video stream 
may inadvertently corrupt the inserted interactive content or simply not re-record it. Also, a 
local subsystem 124 may insert their own content into a video stream, for example, identifying 
call letters. The equipment used to insert this content may inadvertently corrupt the inserted 
interactive content. Thus, a system is needed to ensure the deUvery of the desired interactive 
content to end users. 
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SUMMARY OF INVENTION 

This invention provides technology for ensuring reliable transmission of interactive 
content. This is achieved through the insertion of interactive content codes in the video signal at 
certain points in the path the video takes from the broadcast origination to the customer premise 
equipment. Servers collocated at the broadcasting facility bridge around the hardware that 
destroys the interactive content detect the interactive content codes and use the codes as triggers 
for the blocking, re-inserting, or substituting of interactive content in the transmission path. 

Interactive Content Preservation at the Broadcast Facility 

In a preferred embodiment, this technology is applied to solve the problem where 
interactive content is deleted, destroyed, corrupted, striped out or otherwise modified 
("corrupted") during the transmission path of the broadcast facility due to processing of the video 
stream carrying the interactive content prior to transmission of the video stream, i.e. to a satellite 
or other transmission means. More specifically, interactive content codes preferably comprise 
fields including unique identifiers specifying a single interactive content, or a class of interactive 
content. The identifiers may identify interactive content or other types of content such as 
hyperlinks or URLs. Additionally, interactive content codes may be the interactive content 
themselves. In this embodiment, the interactive content carries the information required to 
provide identification or customization. The interactive content codes also preferably comprise 
timing information indicating the length of time that the interactive content should be inserted 
and conditional information to be used by the local interactive broadcast server to determine 
appropriate action regarding an interactive content. 

In this embodiment, interactive content codes are inserted into a video signal to be 
broadcast to ensure the reUable delivery of the interactive content. The interactive content codes 
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are placed in the video stream either in prepared videos or through an interactive content 
insertion process. Then, an interactive content code detector detects the interactive content 
codes in the video stream at the broadcast faciUty. The interactive content code detector is 
preferably placed in the transmission path before the video stream is transmitted to broadcast 
5 facility hardware that may strip out or corrupt the interactive content codes; for example, in a 
position before video compression broadcast facility hardware. Once the interactive content 
code detector detects a code, an interactive broadcast server determines what action to take, and 
provides data to a data insertion unit accordingly. In one embodiment, the interactive content 
code identifies an interactive content that should be inserted into the video stream to accompany 

10 the broadcast program. In this embodiment, the interactive broadcast server provides the 

identified interactive content to the data insertion unit which inserts the interactive content into 
the video stream at a point in the broadcast facility transmission path subsequent to the broadcast 
faciUty hardware that may corrupt or delete the interactive content. Thus, in this embodiment, 
the present invention ensures that interactive content will be delivered to a local subsystem even 

15 in an environment in which the broadcast facility maintains equipment that may delete or corrupt 
the interactive content. 

An alternative embodiment solves the problem of video content being corrupted in the 
pre-transmission processing path by using interactive content codes placed in portions of the 
video that are guaranteed to be preserved. In this embodiment, the broadcast faciUty may 

20 destroy interactive content and/or interactive content codes as the video stream is being 

generated or at some early point thereafter. Thus, in this embodiment, the interactive content 
codes or interactive content are placed in a portion of the video that is guaranteed by the 
broadcast facility to be transmitted. For example, in one embodiment, the interactive content is 
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placed in the VBI region used for closed captioning. This VBI region has traditionally been 
required to contain closed caption information for transmission to end users and therefore almost 
all broadcast equipment is designed to preserve this VBI region. Thus, in one embodiment, 
interactive content codes are placed in the closed caption region of the VBI and therefore are 
5 preserved by the broadcast facility itself, and an interactive broadcast server is used as described 
above to insert the interactive content into the video stream at a point beyond where broadcast 
facility equipment may delete or corrupt the signal. Thus, this embodiment preserves 
interactivity even if the broadcast facility corrupts the video signal at the video stream generation 
point or early in the transmission path. 

10 Interactive Content Preservation at the Local Subsystem 

Local subsystems also maintain equipment that may destroy interactive content. 
Accordingly, in another embodiment, a local interactive broadcast server is placed at a local 
subsystem to provide reHable interactive content transmission. The local interactive broadcast 
server may also be used to provide content customized for the local subsystem. In a preferred 

15 embodiment, interactive content codes are placed in the broadcast signal as described above. 
The codes are preferably placed in a portion of the broadcast signal that is not typically altered 
by the local subsystem, such as closed caption region of the VBI, also as discussed above. Thus, 
if the local subsystem destroys any interactive content, the interactive content code remains on 
the video signal. The local interactive broadcast server then decodes the interactive content 

20 identification code to determine whether an interactive content should be inserted, replaced, 

passed through or removed from the broadcast signal. Alternatively, an interactive content code 
detector is coupled in parallel to the transmission path of the local subsystem prior to the local 
subsystem equipment that may delete or corrupt the interactive content code. In this 
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embodiment, the interactive broadcast server determines whether to insert, replace, pass through, 
or remove an interactive content responsive to the detected interactive content codes and inserts 
or replaces the interactive content into the transmission path at a point after the equipment that 
may delete the content is coupled. As the interactive content detection technology is placed 
upstream of potential interactive content destroying local subsystem equipment, the content 
codes in this embodiment may be placed in VBI regions other than the closed caption region. 
Local Subsystem Digital Interactive Content Preservation 

In another embodiment, the current invention may be used to preserve content in systems 
v^here the interactive content and video are initially provided in digital format. In this 
embodiment, the interactive content code is extracted by an MPEG or similar decoder at the 
beginning of the transmission path. The interactive broadcast server identifies the nev^ 
interactive content based on the code and provides this content to a data multiplexer that inserts 
the identified interactive content into the transmission path and typically, at the beginning of the 
transmission path (i.e. before the satellite uplink). In addition to satellite, the transmission path 
may include cable or terrestrial (i.e. HDTV). 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram of a prior art interactive content delivery system. 

Figure 2 is a block diagram of an interactive content delivery system v^ith interactive 
content preservation technology. 

Figure 3 is a block diagram of various interactive content codes. 

Figure 4 is an alternate embodiment of an interactive content delivery system with 
interactive content preservation technology. 
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Figure 5 is a block diagram of a closed caption line with interactive content codes. 
Figure 6 is a block diagram of a local subsystem with interactive content preservation 
technology. 

Figure 7 is a flow diagram of the processing of a local interactive broadcast server. 
5 Figure 8 is an illustration of local subsystem data structure. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
1. Interactive Content Preservation and Customization at the Broadcast Facihty 
A. Interactive Content Preservation at the Broadcast Facility 
l^f 10 Figure 2 is a block diagram of an interactive content delivery system with interactive 

I iJ content preservation technology. In a broadcast facility 224, a video source 200 is coupled to a 
ji| video stream generator 212. A broadcast facility 224 may be a broadcast network site, a cable 
Cl network site, or any other site that broadcasts programs and advertisements for view by end users 
with a customer premise equipment CPE 248. The broadcast facility 224 may broadcast directly 
15 to the end user; but may also have its signal relayed through intermediaries such as a local 
l^r subsystem 228 until the signal 227 reaches its final destination of an end user. The video source 
200 may comprise video tapes, UNIX ™ files, or the like, that store programs and 
advertisements as is known in the art. In one embodiment, the video source 201 contains 
interactive content. In this embodiment, the interactive content preparer embeds the interactivity 
20 into the video source 201 (programs, advertisements, etc.) and thus provides the video source 
with embedded interactivity 201 to a broadcast facility 224. The embedding process may be 
accomplished through conventional techniques as is known in the art. Interactive content, as 
used herein and as discussed, comprises interactive applications and/or interactive content codes, 
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and interactive content may serve the same function as the interactive content codes* Interactive 
content may also include portions of an existing interactive application. 

In one embodiment, the interactive content is stored in a vertical blanking interval of the 
program or advertisement, for example in vertical blanking interval ("VBI") Une 15, although 
5 any available portion of the VBI may be used. In this embodiment, a video stream generator 212 
retrieves a video source 200, 201 at a predetermined time and generates a video stream 221 from 
the video source 200. The video stream generator 212 may retrieve the video source 200, 201 in 
accordance with an optional playlist control system 232 that provides program and advertisement 
schedule information. In one embodiment, the video stream generator 212 is a video tape player 
^^f 10 that plays video tape video sources 200, 201 and generates a video stream 221 using a video 
server, such as the Tektronix^^ Profile Video Server, or the like. The video stream 221 may 

•J. 

m comprise video, audio, and data streams, as specified in the NTSC, PAL, and MPEG 
Cj specifications, for example. The video stream 221 is then transmitted to broadcast facility 
C I hardware 216. In conventional systems, the broadcast facility 224 typically modifies the video 
'^f^ 15 stream 221 prior to transmitting the video stream 221 to the transmission facility 220. For 
;3f example, a broadcast facility 224 may choose to compress a video signal that occupies lines 22 
to 262 in each field of a NTSC video signal to occupy only lines 22-212, and use the newly 
available 50 lines to display other information to the viewer, such as a stock ticker, sports ticker, 
or the like. In this embodiment, the equipment 216 used to compress the video signal and insert 
20 the additional information may corrupt the interactive content that was embedded in the video 
source 200. Alternatively, time based corrections (TEC) and frame synchronization equipment 
may be used by the broadcast facility to replace an entire portion of the VBI, in an attempt to 
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clean up the synchronization pulses used to generate a stable image. This equipment may also 
destroy interactive content. 

However, in accordance with the present invention, interactive content codes 300 are 
placed into a video signal to identify interactive content, and the interactive content embedded 
5 video stream 221 is coupled in parallel to an interactive content code detector 204 to detect the 
presence of the interactive content codes 300. In a preferred embodiment, an interactive content 
code 300 is inserted into the video source 201 and/or the video stream 221 (by the data insertion 
unit 208) by an interactive content preparer. As shown in Figure 3, in one embodiment, the 
interactive code 300 is an interactive content identifier 301, which is preferably a string of 

10 symbols (numbers, letters, binary code, a URL, UIC code, or the like) that identifies a particular 
interactive content. The interactive content code 300 may be located in the same location in the 
video source 200, 201 as the interactive content itself, for example, in the VBI of the program or 
advertisement in which the interactive content is designed to be displayed. For example, if an 
interactive content is provided in VBI line 15, the interactive content code 300 is also located in 

15 VBI line 1 5. In this embodiment, the interactive content code detector 204 is preprogrammed to 
examine the VBI line selected for storing the interactive content code 300. An exemplary 
interactive content code detector 204 would be a VBI line reader as is known in the art, for 
example, the Norpak^^ TTX-742 VBI Reader and which can be modified to read the specific 
VBI line that contains the interactive content identifier 501. 

20 In an alternate embodiment, the interactive content identifier 301 is located in a different 

location than the interactive content. As the interactive content identifier 301 is typically smaller 
in size than an interactive content, this embodiment allows the interactive content code 300 to be 
placed in a wider variety of regions in the video source 200, 201. For example, the interactive 
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content code 300 may be placed in a field that may have less bandwidth because of its use to 
store other information, but is less likely to be deleted than other fields of the VBI; for example, 
in the closed caption region. As the closed caption region is not intended for large quantities of 
data, there may not be sufficient bandwidth to store an entire interactive content. However, in 
5 this embodiment, the smaller interactive content code 300 is stored in this region without 

interfering with the closed captioning text and can be transmitted safely, as is discussed in more 
detail below. Altematively, the interactive content code 300 is encoded into a parallel data 
stream that may be broadcast in the transmission path, such as in a separate audio stream, a 
separate closed caption region (for a digital broadcast signal), a separate address stream, or any 
;^- J 10 other stream separate fi:om the stream used to convey the interactive content. For example, some 
IjJ broadcasts broadcast a separate audio stream (e.g., SAP) along with their programming. In this 
m embodiment, the interactive content codes 300 are embedded in the separate channel, and the 
Cl interactive content code detector 204 is coupled to the separate audio stream to detect the 

.as:;, 

}^"^ interactive content code 300. After a code 300 has been detected, the data insertion unit 208 
1 5 inserts the corresponding interactive content into the broadcast signal, typically at a point in the 
transmission path after the broadcast facility equipment that may destroy the interactive content 
is coupled to the broadcast path. 

In an altemate embodiment, the interactive content code 300 is an interactive conditional 
code 303. An interactive conditional code 303 may provide conditional instructions for inserting 
20 interactive content into the modified video stream 225, as discussed below. In this embodiment, 
the interactive code detector 204 reads the interactive conditional code 303 present in the video 
stream 221. In another embodiment, the interactive content code 300 is contained in an existing 
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interactive content. In this case, the processing of the interactive content code may result in the 
blocking, substituting, or passing through of the original interactive content. 

Once the interactive content code detector 204 has read the appropriate section of the 
video source 200, 201, the interactive content code 300 is transmitted to interactive broadcast 
5 server 206. The interactive broadcast server 206 decodes the interactive content code 300 and 
determines instructions for providing an interactive content. For example, if the interactive 
content code 300 is an interactive content identifier 301, the interactive broadcast server 206 
decodes the data to identify the interactive content identifier 30L The interactive broadcast 
server 206 may be a UNIX server, a Windows NT server, or any other similar server executing a 

10 process designed to perform the functionality herein. The interactive broadcast server 206 is 
coupled to a database 244 of interactive content, and retrieves the identified interactive content 
firom the database 244. Then, the interactive broadcast server 206 provides the interactive binary 
data 222 to a data insertion unit 208 to have the data insertion unit 208 insert the interactive 
content into corrupted video stream 225. Video stream 225 is the video stream 221 as corrupted 

15 by the broadcast facility hardware 216, and thus may or may not retain the interactive content of 
video stream 221. The data insertion unit 208 is preferably a data insertion unit as is known in 
the art, such as the Norpak TES-3 Multi Standard Encoder. The data insertion unit 208 
overwrites or modifies any ongoing interactive content playback in the video stream 225 to 
generate an interactive embedded video stream 226. This interactive embedded video stream 

20 226 is sent to a transmission facility 220, which then transmits the interactive embedded video 
stream 226 to the local subsystem 228 or CPE 248. Thus, in accordance with the present 
invention, interactive content can be reliably deUvered to a CPE 248. 
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In one embodiment the interactive broadcast server 206 immediately inserts the 
interactive content as long as the interactive content identifier 301 is detected in the video signal. 
In an alternate embodiment, an enhanced interactive content identifier 302 also provides timing 
information 306. Timing information 306 comprises start and stop information, duration 
5 information, or start and duration information, or other information useful in determining when 
to insert an interactive content. Thus, in this embodiment, the interactive broadcast server 206 
retrieves the interactive content in accordance with the timing information 306. For example, in 
an embodiment in which the timing instructions provide start and duration information, the 
interactive broadcast server 206 has the interactive content inserted at the time indicated by the 
1? 10 start time and has the insertion stopped after a period of time corresponding to the duration 
ijl information. This embodiment ensures that the interactive content are properly synchronized 
yl with the programs and^r advertisements to which they relate. 

C I In one embodiment, the interactive content code 300 is placed at the beginning of a 

l^f program and the timing information 306 is a duration value that indicates how many seconds 

15 remain in the program. Thus, the interactive content is started when the interactive content code 
{^l 300 is detected and is stopped when the duration value expires. In an alternate embodiment, 
interactive content codes 300 are inserted throughout the program, and the timing information 
306 includes a duration value that indicates how long before the next interactive content code 
300 will appear. For example, an interactive content code 300 may be inserted every 4 seconds, 
20 with a duration of 5 seconds. If after 5 seconds the interactive content code detector 204 has not 
detected a new interactive content code 300, the program is considered to have ended and the 
interactive content is stopped. This implementation is useful if the durations of programs are 
unknown. In a digital environment, certain digital broadcast formats already provide time 
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stamps, and therefore, in these formats the interactive content codes 300 can synchronize the 
insertion of interactive content through the use of the pre-existing time codes. Alternatively, as 
digital broadcasts are comprised of and are nxmibered by frames, the interactive content codes 
300 can specify the insertion of interactive content by frame number, e.g., "insert appUcation X 
5 at frame 596". 

In a further embodiment, the broadcast facility generates digital video signals, and the 
interactive content code detector 204 is an MPEG decoder, and identifies the corresponding 
interactive content based on the decoded interactive code. Then, the interactive content code 
detector 204 provides this content to a data multiplexer or similar device that inserts the 

10 identified interactive content into the transmission path. For example, immediately before a 
satellite uplink, or after a point in the transmission path that contains equipment that might 
corrupt or otherwise alter the interactive content. The transmission path, however, may also be 
cable or other terrestrial, i.e. HDTV. 

The interactive content code detector 204 is typically coupled to a point in the 

15 transmission path prior to the point at which broadcast facility equipment 216 may corrupt the 
interactive content code 300. The data insertion unit 208 is typically coupled to a point in the 
transmission path after the hardware 216 is coupled, to ensure the interactive content to be 
inserted is not deleted. Thus, the present invention enables rehable transmission of interactive 
content to an end user. In one embodiment, the detector 204 is coupled to the transmission path 

20 immediately after a video stream is generated, for example, after a video stream generator 212 or 
a video source generator is coupled to the transmission path, to ensure that the content code 300 
will be detected before being transmitted through hardware that may delete the interactive 
content code 300. In a further embodiment, the DIU 208 is coupled at a point in the transmission 
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path immediately before transmission, for example, immediately before the transmission facility 
220, to ensure that the interactive content will be broadcast to the local subsystem 228. If the 
broadcast facility does not contain equipment that will destroy the region of data that contains 
the interactive content code, the interactive content code detector 204 can be coupled to the same 
5 point in the transmission path as the data insertion unit 608, which will be after the point at 
which hardware that will corrupt the interactive content is coupled to the transmission path. 
B. Interactive Content Preservation Through Guaranteed Field Selection 
In one embodiment, interactive content is preserved through the placement of interactive 
content codes 300 in VBI regions that are preserved by the broadcast facility. Figure 4 is an 

10 altemate block diagram of a broadcast facihty 428 in accordance with this embodiment of the 
present invention. In this embodiment, the video having embedded interactive content 201 is 
input to format conversion equipment 404 of a broadcast faciUty 428, The format conversion 
equipment 404 converts the video into a format compatible with the broadcast facility's video 
stream generator 212. As different broadcast facilities use different format conversion 

15 equipment 404, the interactive content preparer typically prepares video in a single format and 
allows the broadcast facility to convert the video into the appropriate format. However, format 
conversion equipment 404 often strips or deletes vertical blanking intervals fi-om a video 400, 
thereby unintentionally removing interactive content. Additionally, some broadcast faciUties 
broadcast digital signals. In this example, if the video source is analog, then the broadcast 

20 facility must convert the analog signal mto a digital signal. This process also may strip or delete 
the VBI or certain VBI lines. Thus, in an embodiment in which interactive content is stored in 
the VBI (or VBI line), the interactivity may be deleted at this point in the video transmission 
process. Additionally, the video stream generation equipment 212 may also delete or remove 
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interactive content or codes stored in the VBL If interactivity is deleted at either of these points, 
the interactive content code detector 204 will be unable to detect the interactive content code 
300, 

In accordance with the present invention, an interactive content code 300 is placed the 
5 closed caption VBI line e.g. Kne 2L As mentioned above, the closed caption line 21 is 

traditionally preserved throughout transmission. Thus, the broadcast facilities may use special 
equipment, referred to as a "bridge" (not shown), that explicitly preserves the data in the closed 
caption line. In this embodiment, the interactive content provider embeds the interactive content 
code 300 into the closed caption line of the video using a closed caption encoder 408 to create 

10 video with embedded interactivity 400. The closed caption encoder 408 is a conventional video 
encoding device such as the Norpak^™^ TES-3 or the EEG closed caption encoder, that is 
designed to encode data in the closed caption line. Once the interactive content code 300 is 
placed in the closed caption Une, the broadcast facility 428 itself will ensure that the content code 
300 is preserved throughout the transmission path of the broadcast facility. Thus, the interactive 

1 5 content code detector 204 will be able to detect the interactive content code 300 by reading the 
closed caption line of the signal 221, and then the interactive broadcast server 206 will be able to 
insert the corresponding interactive content. Of course, the interactive content code 300 may 
identify an appUcation, an embedded program or advertisement, may provide timing information 
306, and may provide conditional information 308, as discussed below, to provide on-the-fly 

20 adjustments. 

Figure 5 illustrates contents of the closed caption line. As illustrated, the closed caption 
line comprises two fields, field 1 and field 2. Field 1 has four subfields, closed caption 1, closed 
caption 2, text 1, and text 2. Field 2 has subfields closed caption 3, closed caption 4, text 3, text 
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4, and XDS. The closed caption 1 field stores the closed caption information that is displayed on 
a user's television when the closed caption feature is enabled, i.e., the text of the dialogue 
occurring in the program or advertisement. The other closed caption fields are used to display 
alternate languages in the closed caption format. The Text 1, Text 2, Text 3, Text 4, and XDS 
fields are not specifically required to carry any data; thus, they may be used to carry commercial 
data such as interactive content identifiers 300 or interactive content, XDS, for example, is 
specifically designed to carry auxiliary data, and current law requires the transmission of V-chip 
data in that field. Thus, an interactive content provider can encode its data in any of these fields. 
It is preferable to store data in Field 1, as that field is the only field that is required to have closed 
caption information by law and is therefore almost always preserved by the broadcast facility 
itself However, either field may be used as it is difficult to accidentally delete one field and not 
the other. Thus, most equipment that preserves field 1 of the closed caption region will by 
necessity preserve field 2. Although the above description refers to inserting interactive content 
into the closed caption line, any VBI line or other region that is preserved throughout the 
transmission path of a video stream may be used in accordance with the present invention. It is 
fiirther anticipated that federal law or other regulatory regimes may require preservation of other 
VBI lines for other purposes, and the present invention may be readily extended to those other 
lines. 

In this embodiment, the interactive content code detector 204 and the data insertion unit 
208 may be coupled to approximately the same point in the transmission path of the broadcast 
facility 224 because the code 300 is guaranteed to be preserved by the broadcast facility 
equipment. Thus, in this embodiment, the interactive code detector 204 reads a preserved line 
that has an interactive content code 300, and the data insertion xrnit 208 inserts the interactive 
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content into a different region that may not have been preserved at the same point at which the 
code 300 is detected. In this example, the interactive content code detector 204 and DIU 208 are 
coupled to a point in the transmission path after which broadcast facility equipment that may 
destroy the interactive content is coupled to the transmission path, to ensure that the inserted 
5 interactive content is preserved. 

In an embodiment in which references to the interactive content itself (such as a URL) 
are encoded into VBI line 21, the interactive content code detector 204, interactive broadcast 
server 206, interactive content 208, and interactive content database 244 may be optionally 
eliminated. If a URL is used, the interactive content code detector 204 decodes the URL, 
CI 10 connects to the database (located remotely or locally) identified by the URL through a 

communication facihty such as a modem, and retrieves the associated interactive content for 
jlj insertion by the DIU 608. As the interactive content is ensured of its transmission by its 
n placement in the closed caption line, then reinsertion is not necessary. However, if on-the-fly 
CI replacement is desired, or if the interactive content is placed elsewhere in the video source 200, 
Cl 1 5 201 , then the above equipment would be used to provide the above-described functionality. 

Thus, this embodiment of the present invention provides further guarantees that an interactive 
content will be preserved throughout a transmission path until reaching the CPE 248. 
C. Customization of Content at Broadcast Facility 

Referring to Figure 3, the system of the present invention may also provide on-the-fly 
20 content adjustment. In this embodiment, an interactive conditional code 303 is provided with an 
interactive content. In an embodiment with video sources embedded with interactivity 201, an 
interactive content is embedded in the video source 201 and an interactive conditional code 303 
is embedded as well. The interactive conditional code 303 preferably specifies an interactive 
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content identifier 301 and an OPT field 308. The interactive content identifier 301, as discussed 
above, can refer to either an interactive content or a video source embedded with an interactive 
content 201 . The OPT field 308 specifies whether an interactive content may be replaced by the 
interactive broadcast server 206. If an OPT field 308 indicates that an interactive content cannot 
5 be replaced, then the interactive broadcast server 206 knows that no new interactive content is 
available for that particular video 200, Thus, the use of the OPT field 308 allows the interactive 
broadcast server 206 to immediately determine whether an interactive content is available for 
replacement. If the OPT field 308 indicates that an interactive content may be replaced, then the 
interactive broadcast server 206 determines whether or not to replace the existing interactive 
C l 10 content with a different interactive content. For example, if an interactive content is embedded 
J into a video source 201, but the interactive content supplier or the program source determines 

5- 

;^ a 

II J that a different version should be broadcast, the different version can be stored in database 244, 
Q and the interactive broadcast server 206 can replace the existing version of the application with 
Cl the newer version as identified in the interactive content code 300. For example, an interactive 
5i 1 5 content that relates to a news story may be replaced with a more current version of the interactive 
f content as the news story changes merely by replacing the interactive content at the broadcast 
facility 224. Thus, the interactive content provider can easily make on-the-fly adjustments to 
interactive content. In this embodiment, the interactive content provider embeds the original 
video source 200, 201 with an OPT field specifying that the interactive content can be replaced, 
20 along the interactive content identifier. However, the use of an OPT field 308 also allows the 

interactive content preparer to ensure that interactive content are not replaced by a third party not 
under the control of the interactive content preparer. The on-the-fly embodiment of the present 
invention may also be performed without an OPT field. In this embodiment, the interactive 
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broadcast server 206 automatically inserts a latest version of an interactive content upon receipt 
of an interactive content identifier. The OPT field 308 may also be used to specify conditions for 
replacement. For example, for a contest interactive content, the field 308 could specify that 
interactive content be broadcast only if the local subsystem has legalized such contests, as 
5 discussed in more detail below. In this case, OPT field 308 will have a nimierical value that can 
be compared to the values known to a local server. The local server determines whether the 
application should be blocked, passed through, or replaced from the comparison. 
II. Interactive Content Preservation at the Local Subsystem 
A. Interactive Content Preservation 

10 Figure 6 is a block diagram of a local subsystem 628 with interactive content preservation 

technology. In this embodiment, the local subsystem 628 receives a broadcast signal 601 fi'om a 
broadcast facility and relays the signal 601 to a CPE 248. The local subsystem 628 is typically a 
local affiliate of the broadcast facility; but the local subsystem 628 could also be a cable headend 
that receives a broadcast signal 601 firom a cable provider or broadcast facility, or local affiliate, 

15 or a digital satellite subsystem that receives a signal 601 fi:om a local affiliate, a broadcast 
facility, or a cable headend. The broadcast signal 601 contains either interactive content, 
interactive content codes 300, or both. End users receive the broadcast signal 601 on their CPE 
648, which may be a television, set top box, or satellite receiving system. The interactive content 
is displayed along with, or over, the portion of the broadcast signal video. 

20 In accordance with the present invention, interactive content codes are placed in the 

broadcast signal to ensure reliable delivery of interactive content through the local subsystem 
628 to the CPE 648. In one embodiment, the interactive content codes 300 are not located in 
closed caption VBI line, thus, they may be modified or deleted by the local subsystem 628. As 
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shown in Figure 6, a receiver 612 at the local subsystem 628 receives the broadcast signal 601. 
Then, the local subsystem 628 uses local subsystem equipment 616 to perform different 
operations on the received signal 602, For example, the local subsystem 628 may store a 
received signal 602 to rebroadcast the signal 602 at a later time. Further, the local subsystem 
5 628 may insert graphics, such as a local station ID, into the signal 602 prior to retransmission. 
Alternatively, the local subsystem 628 may insert a local commercial into the video stream 602 
in a place reserved by the broadcast facility for local insertion. All of these operations may 
inadvertently destroy the interactive content or a previously inserted interactive content code by 
modifying or deleting the component of the video signal 602 carrying the interactive content. 
C-f 10 In accordance with the present invention, an interactive content code detector 604 is 

f:{ coupled in parallel to video signal 602 to detect the inserted interactive content codes 300. The 
J i interactive content code detector 604 may be coupled directly to the output of the receiver 612, 
Zl or may be coupled further along the transmission path. The interactive content code detector 604 
Q is preferably coupled to the video signal 602 prior to the video signal 602 being transmitted to 
- i 1 5 any local subsystem equipment 616 that may destroy the interactive content code 300. The 

interactive content code detector 604 detects the interactive content code 300 and provides the 
code 300 to a local interactive broadcast server 606, which then identifies the interactive content 
and causes the data insertion unit 608 to reinsert interactive content into the modified video 
stream 607 at a point in the transmission path after which equipment that may corrupt the 
20 interactive content is coupled to the transmission path. The operation of the interactive content 
code detector 604 in this embodiment of the present invention is substantially similar to the 
embodiment of the interactive content code detector 204 described above with respect to Figure 
2. In one embodiment, the data insertion imit 608 is coupled to a point in the transmission path 
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immediately before the video signal is broadcast out to the CPEs 648, to ensure that no 
equipment from the local subsystem will corrupt the inserted interactive content. 

Altematively, in one embodiment, the interactive content provider stores multiple 
interactive content codes 300 in different fields of the video signal 602 prior to being broadcast 
5 to the local subsystem 628 to ensure that one of the interactive content codes 300 is transmitted 
successfully through the local subsystem 628. As different local subsystems 628 may 
independently preserve different fields, this method helps ensure that at least one of the 
interactive content codes 300 present in a broadcast signal is not modified or deleted by the local 
subsystem 628 and can be detected by the interactive content code detector 604. 

10 In an embodiment in which digital applications are to be inserted into a broadcast digital 

video stream, the data insertion unit 608 comprises MPEG or other digital format insertion 
capabilities, as is known to those of ordinary skill in the art. For example, for a MPEG 1 or 2 
stream, the interactive broadcast server 606 transmits the interactive content to a local subsystem 
multiplexer that is generating the MPEG stream. The local subsystem multiplexer typically has a 

15 serial or TCP/IP port to receive the interactive content bit stream. Upon receipt of the interactive 
content bit stream, the multiplexer adds the interactive content into the digital video stream in 
accordance with its normal operation. The multiplexer can also remove and replace interactive 
content when appropriate. If the digital stream is Serial Component Digital Video, (a packet- 
based digital video stream) as specified in the SMPTE 259M-C standard, a Norpak encoder 

20 would be used to add the interactive content by replacing the existing packets with the new 

packets transmitted to it by the local interactive broadcast server 606. Alternatively, if a digital 
version of an interactive content has been broadcast to an analog subsystem 628, the local 
interactive broadcast server 606 can swap out the digital application with a lower bandwidth 
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analog application. If an analog version of an interactive content has been broadcast to a digital 
subsystem 628, the analog version must be converted into a digital program or a digital program 
must be swapped in place of the analog version. If the analog version is to be digitized, a 
software analog to digital converter must be used to convert the analog version into a digital 
5 program that accords with the local subsystem's protocols. In one embodiment, the local 

interactive broadcast server 606 supports multiple application delivery to the subsystem 628. For 
example, a cable headend subsystem 628 may dehver both analog and digital streams to allow 
homes with digital set tops to receive higher quality interactivity while homes with analog may 
still receive the simpler interactivity. In this embodiment, the local interactive broadcast server 

Cl 10 606 selects both the richer and simpler versions of the interactive content and inserts them into 

"f the respective signals. 

3 J In an enibodiment in which the interactive content codes 300 are placed in a field that is 

?l preserved by the local subsystem hardware, for example, in the closed caption line, the 
CI interactive content code detector 604 is coupled to the transmission path to detect the interactive 
1 1 1 5 content code 300 and then have the data insertion unit 608 insert the interactive content into the 
video signal at a point in the transmission path after which local subsystem hardware that may 
corrupt the signal is coupled to the transmission path. In one embodiment, the interactive 
content code detector 604 and the data insertion unit 608 are coupled to the same point in the 
transmission path. 

20 Thus, this embodiment of the present invention ensures the successfiil transmission of 

interactive content through the local subsystem 628. Of course, the interactive content identifier 
300 may identify an appUcation, an embedded program or advertisement, may provide timing 
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information 306, and may provide conditional information 308, as discussed above, to provide 
the benefits and advantages discussed above. 
B. Interactive Content Customization 

In an altemate embodiment, the local subsystem 628 also provides customized content. 
In this embodiment, interactive content are targeted for a local subsystem 628, providing for a 
much richer interactive content experience. Conditional fields are provided to allow the local 
subsystem to perform the appropriate action for the local subsystem. For example, depending on 
the conditional field and the current configuration of the local subsystem, the local subsystem 
may disable an existing interactive content, replace an existing application v/ith a customized 
application, pass through an^existing application, or insert an interactive content. 

In one embodiment, the local interactive broadcast server 606 inserts a corresponding 
interactive content into the video signal 607 for either customization or reinsertion purposes. 
However, in this embodiment, the corresponding interactive content may be a custom interactive 
content. For example, if a local subsystem 628 determines that the demographics that constitute 
the viewing region would prefer a different version of an interactive content, the interactive 
content provider will provide a version of the interactive content suited to the demographics of 
the local subsystem 628. Alternatively, if the local subsystem 628 has a different bandwidth 
capability than the bandwidth capabiUty with which the existing interactive content was intended 
to be used, the local interactive broadcast server 606 can substitute an interactive content 
designed for the bandwidth capability of the local subsystem 628 for the existing interactive 
content. For example, if the majority of all end users have low-bandwidth connections, the 
interactive content provider will typically provide an interactive content designed to be 
transmitted over the low-bandwidth connection. However, if a specific local subsystem 628 is a 
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high-bandwidth digital cable subsystem, then, in accordance with the present invention, the 
digital local interactive broadcast server 606 replaces the low-bandwidth interactive content with 
a digital version of the interactive content, allowing the local subsystem 628 to take advantage of 
its superior network infrastructure. 
5 In an alternate embodiment, an interactive conditional code 303 is provided in the video 

signal 601. As discussed above, the interactive conditional code 303 comprises an OPT field 
308* The OPT field 308 is set to designate whether an interactive content may be replaced. 
Additionally, the OPT field 308 may provide information to be interpreted locally for deciding 
with what content to replace the existing interactive content. For example, the OPT field 308 
10 may have a flag that indicates the type of content being transmitted, a ticker, a contest, an 

\ § 

/^ J advertisement, or any other type of content or whether the content should be blocked. This 
Jl| content type information can be used by the local subsystem to determine an action. For 
J I example, the local subsystem may prohibit advertisements within a program unless they derive 
1^1 revenue from the advertisement. Thus, if a flag indicates an advertisement is being transmitted 

15 within a program, the local subsystem may disable the ^plication as discussed below. The 
I";' interactive content provider typically sets the OPT field 308 of the interactive conditional code 
303 to indicate that the interactive content can be replaced, however, use of the OPT field 308 
also allows an interactive content provider to ensure that certain appUcations are not replaced or 
modified by a local subsystem 628. If the interactive content can be replaced, the interactive 
20 content provider sets the parameter for replacing the interactive content in the OPT field 308. 
In a preferred embodiment, the interactive content codes 300 may be inserted, as 
discussed above, into the closed caption region of the VBI, as a code in the video, or encrypted 
as a unique identifier to prevent "ad skipping" appHcations from being developed. As discussed 
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above, if the interactive content codes 300 are originally inserted into the closed caption region, 
the interactive content codes are ensured to be transmitted to the interactive content code detector 
604. 

A preferred method of processing interactive content codes 300 is illustrated in Figure 7. 
5 In this embodiment, as discussed above, there are four main actions a local server 606 can take: 
disable 700 an existing interactive content, insert 704 an application, pass through 708 an 
existing appUcation, and replace 712, 716, 720, 724 an existing appHcation with a customized 
apphcation. First, the server 606 at the local subsystem 628 determines 728 whether an 
interactive content is present in the received signal 601 . In one embodiment, the broadcast 
51 10 facility does not insert appUcations into the video stream 601, allowing the local subsystems 628 

instead to provide the insertion. In this embodiment, a interactive content identifier 300 is 
:|; provided with program or advertisement material to identify the application to be inserted into 
Q the video stream 607. Thus, this embodiment eliminates the requirement of having an interactive 
51 broadcast server 104 at the broadcast facility. If the interactive content identifier 301 is inserted 
CI 15 into the closed caption region of a video source 200, there may be no interactive equipment 
present at a broadcast facility at all, as the broadcast facility does not have to ensure the 
preservation of the codes 301 with the interactive content preservation technology described 
above. 

If an interactive content is not present in the signal 601, either because the broadcast 
20 facility did not insert an appHcation as discussed above or because the broadcast facility deleted 
or removed the interactive content, then the server 606 determines 744 if there are multiple 
appUcations in database 644 corresponding to the interactive content code 300. Figure 8 
illustrates an embodiment of database 644. Database 644 maintains interactive content and 
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database 646 maintains video sources 201 with embedded interactivity. Either or both can be 
used in accordance with the present invention. The databases 644 are preferably organized by 
interactive content identifiers to allow the local interactive broadcast server 606 to quickly locate 
the information and any corresponding applications released to an interactive content. In one 
5 embodiment, the database 644 maintains a coxmt field for each interactive content that indicates 
the number of different versions of an interactive content that are present in the database 644. A 
count of 2 indicates that an additional version of an interactive content is stored in the database 
644. The server 606 locates the count field corresponding to the interactive content code being 
analyzed and examines the count field to determine whether there are multiple versions of an 
Gl 10 interactive content. Ifthere are not multiple applications, then the server 606 retrieves the 

! i % 

^ J corresponding interactive content and provides it to the data insertion unit 608, which inserts 704 
the corresponding application into the video stream 601 . The resultant embedded stream 609 is 
transmitted to the CPE 248. 
Q If an interactive content is present in the signal, the server 606 determines 732 whether 

Cl 15 the subsystem 628 has blocked the application. In one embodiment, the local server 606 

maintains a list of those applications which are blocked by the local subsystem 628 in database 
644. As shown in the example of Figure 8, these interactive content are identified in the 
database 644 as being blocked for transmission. Thus, in this embodiment, the local subsystem 
628 has control over whether an interactive content is provided to an end user by modifying the 
20 list maintained by the server 606. This allows local subsystems 628 to control the content of 
their transmissions in accordance with viewership preferences, legal issues, or the like. As 
discussed above, the OPT field 308 specifies the type of interactive content (for example, a 
contest). In this embodiment, the local interactive broadcast server 608 reads the OPT field 608 
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and compares the OPT field 308 with local content parameters to determine the broadcast of the 

1 

application is permitted. The server 606 will look up a corresponding contest field in its 
database to determine its status. For example, if prize awarding contests are illegal in a 
subsystem, a contest interactive content will be disabled. This embodiment also allows the local 
5 subsystem 628 to gain revenue by retaining a percentage of any revenue generated by a 

interactive content. For example, if an interactive content is a web site link at which an end user 
may make a purchase, the local subsystem 628 can require the web site host to pay the local 
subsystem 628 a percentage of the revenue gained in exchange for permitting the interactive 
content to be transmitted to the CPE 248. In this embodiment, if no contractual arrangement has 
Cl 10 been reached with the web site host, a flag on the database 644 will indicate that the local 
^ interactive broadcast server 606 should disable the corresponding interactive content. 
III If a local subsystem 628 has indicated that an interactive content is to be blocked as 

r I discussed above, the server 606 disables 700 the interactive content. Altematively, the server 
CI 606 determines whether an interactive content may be transmitted on the subsystem 628 due to 

Wis 

j^^ 15 bandwidth considerations. For example, if an interactive content is designed to require more 
'r^ bandwidth than a local subsystem 628 can provide, the server 606 also disables 700 the 
interactive content. Thus, in this embodiment, and as shown in Figure 8, the server 606 
maintains the bandwidth data of the local subsystem 628 in the database 644, and bandwidth data 
of the interactive content is either provided explicitly in a field of the interactive content code 
20 300 or the bandwidth data is available through analysis of the appUcation itself If there is 

insufficient bandwidth, the server 606 disables 700 the application. Stripping the data from the 
signal may disable the interactive content. In cases where regulations forbid the stripping of 
data, the display of the original application may be disabled by inserting a new data field which 
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will be used as a flag by the CPE 248 to ignore the transmitted data. If a lower bandwidth 
version exists, this condition is processed as discussed below. 

If the server 606 determines that the interactive content should not be disabled, the server 
606 also determines 740 at this point whether there are multiple versions of the interactive 
5 content in the database 644. If there are not, then the server 606 passes 708 the interactive 
content provided with the signal 601 to the CPE 248. 

If at either decision point 740 or 744 the server 606 determines that there are multiple 
applications stored in the database 644 the server 606 determines 748 whether a local customized 
application is present in the database 644. As discussed above, a local customized apphcation is 
CI 10 an application that is designed for the geographic or demographic viewership of a local 
/J J subsystem 628. In a preferred embodiment as shown in Figure 8, the database 644, 648 has 
identification fields that identify the nature ofdifferent applications in the database. For 
example, in Figure 8, interactive content ID222A is a local version of ID222, and ID174A is a 
Zl digital version of ID 174. Both ID222 and ID 174 are interactive content identifiers that would be 
Q 1 5 present in interactive content code 300 or would be part of the interactive content itself Thus, 
the server 606 will examine the database entry for a specific application, and if an identification 
field indicates that a corresponding local customized apphcation is present, the server will next 
determine 750 if there are multiple transmission streams in the subsystem 628. For example, 
some cable headends provide both analog and digital video streams. In this embodiment, the 
20 server 606 may maintain a field that indicates whether or not the subsystem 628 provides 

multiple transmission streams. If there are not multiple streams, the server 606 replaces 712 the 
existing application with the local customized apphcation as identified in the database 644, 648. 
The server 606 replaces an application by providing the new corresponding interactive content to 
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the data insertion unit 608 to overwrite the existing interactive content with the new 
corresponding interactive content. If there are multiple streams, the server 606 places 754 the 
different versions of the local customized appUcation into the appropriate streams for broadcast. 
Thus, the present invention allows targeted interactivity to be provided to end users, making the 
5 interactive content more effective and attractive to the end users than in conventional systems. 
If the server 606 determines that there are no local customized applications in the 
database, the server 606 also determines 752 whether the local subsystem 628 had multiple 
transmission streams. If there are multiple transmission streams, the server 606 places 720 the 
corresponding high bandwidth appUcation identified in the database 644 in the high bandwidth 

10 stream and places or allows to pass through a low-bandwidth application in the low bandwidth 
stream. This allows a local subsystem 628 to tailor its interactive content to end users based on 
the end users's capabilities. If there is only one transmission stream, and in an embodiment in 
which a low bandwidth application is the application version transmitted on video signal 601 
(which is the most common case to ensure the most universal applicability of an interactive 

15 content), the server 606 determines 756 whether a high bandwidth interactive content is present 
in the databases 644 that corresponds to the interactive content code being analyzed. If there is, 
the interactive broadcast server 606 replaces 712 the low bandwidth interactive content with a 
high bandwidth version of the appUcation to take advantage of the high bandwidth capabilities of 
the local subsystem 628. If there is not a high bandwidth application present in the system, then 

20 the server 606 looks for other customized versions of the appUcations and processes 724 those 
appUcations responsive to their conditions. The above order of processing is merely exemplary, 
as decisions can be made in different orders within the scope of the present invention. 
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The foregoing describes in details the features and benefits of the present in various 
embodiments. Those of skill in the art will appreciate that present invention is capable of various 
other implementations that operate in accordance with the foregoing principles and teachings. 
For example, the arraagement and organization of the broadcast and local subsystem facihty 
hardware may differ and the interactive preservation technology can be located in the appropriate 
position in accordance with the teachings described herein to achieve the described results and 
benefits. Certainly, the names of the various entities may be changed without impacting their 
fimctional operations. Accordingly, this detailed description is not intended to limit the scope of 
the present invention, which is to be understood by reference the claims below. 
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